Accuracy of the urine point-of-care circulating cathodic antigen assay for diagnosing Schistosomiasis mansoni infection in Brazil: A multicenter study

ABSTRACT Background: The World Health Organization recommends a market-ready, urine-based point-of-care diagnostic test for circulating cathodic antigens (CCA) to determine the prevalence of S. mansoni. This study evaluated the performance of the URINE CCA (SCHISTO) ECO TESTE® (POC-ECO), which is currently available in Brazil. Methods: Residents from eight sites with different prevalence estimates provided one urine sample for POC-ECO and one stool sample for Kato-Katz (KK) and Helmintex® (HTX) testing as an egg-detecting reference for infection status. Results: None of the study sites had significantly higher POC-ECO accuracy than KK. Conclusions: POC-ECO is not currently recommended in Brazilian schistosomiasis elimination programs.

In Brazil, approximately 1% of the population is infected with Schistosoma mansoni, mainly in areas with unsafe water supply and poor sanitation 1 . The Brazilian government has endorsed successive World Health Assembly resolutions and World Health Organization recommendations to eliminate schistosomiasis, including the WHO roadmap 2021-2030 2 . In response to the country's ecoepidemiological characteristics and public health policies, the Ministry of Health (MoH) recommends communitywide active search and treatment of infection carriers at the primary care level in low-risk endemic areas, which account for most sites 3 . Currently, this includes regular stool examinations using the KK diagnostic method 4 , followed by administration of praziquantel for egg-positive cases in all age groups if there is no individual contraindication. This test-and-treat regimen is not appropriate for these situations because routine KK (one sample and two slides) cannot detect infected cases with a low egg burden. Studies using robust egg detection methods as a diagnostic reference have shown that a relatively high proportion of egg-positive cases may be missed by routine KK and thus go untreated 5,6 .
The WHO recommends the use of a market-ready, urinebased point-of-care circulating cathodic antigen (POC-CCA) assay to determine the prevalence of S. mansoni infection 7 ; POC-CCA is considered more sensitive than the routine KK and is easy to handle at a low cost 8 . POC-CCA has been available in Brazil since 2017 under the name URINE CCA (SCHISTO) ECO TESTE® (short: POC-ECO). The test is manufactured by ECO Diagnóstica Ltda. under license from Rapid Medical Diagnostics (RMD), South Africa, and is approved by the Brazilian Health Regulatory Agency (Anvisa) for use as an auxiliary diagnostic tool in clinical practice. However, recent studies from Africa and Brazil suggest that the sensitivity and specificity of POC-CCA are lower in areas of moderate and low endemicity than in areas of high endemicity, and there is increasing evidence that it can lead to false positive results 9 Therefore, the MoH commissioned a multicenter research study to evaluate the performance of the product in the conditions of the Brazilian Schistosomiasis Control Program (PCE), particularly in low-risk settings 10 . This paper describes the results of this study on the accuracy of POC-ECO at representative sites with different levels of endemicity, following the same core protocol. The study protocol was approved by the Ethics Committee of the Oswaldo Cruz Institute-Fiocruz (CAAE: 82469417.8.0000.5248).
Eight sites were selected for the study because the prevalence was already known based on routine municipality-level KK results between 2008 and 2017 (Figure 1). No treatment interventions were performed at any study site in the 12 months before the study. After obtaining written consent from the health authorities in each municipality, research teams contacted study sites to recruit residents aged two years or older. Individuals with selfreported pregnancy or clinical signs/symptoms of acute illness, severe chronic diseases, and praziquantel treatment within the past 12 months were excluded. Those who agreed to provide free, informed consent were given collection vials and asked to provide ~10 mL of midstream morning urine and ~50 g of stool (one sample each) to be refrigerated until the next day. Samples were then collected by the research teams and transported in cool containers to a local laboratory, where they were immediately processed. Urine samples were tested using POC-ECO according to the manufacturer's instruction leaflet issue 001/2018, which was included in the kit 10 . The outcome was graded by G-scores, as described by Casacuberta-Partal et al. 11 . Batch 180907091 was used at all sites except Palmital, where batch 201806011 was used for operational reasons. Stool samples were processed and tested by the KK and Helmintex® (HTX) methods, as described by Graeff-Teixeira et al. 9 . Subjects who tested positive for S. mansoni eggs by the KK or HTX method and those who showed soiltransmitted helminths by the KK method were referred to a local health facility for treatment, as described by MoH 3 .
Research teams aimed at testing 300 participants per site, based on recommendations from Bujang and Adnan 12 for evaluating the sensitivity and specificity of the diagnostic tests with a statistical power of 80% and a significance level of 5% (α = 0.05). Every effort was made to include as many participants as possible. No randomization procedure was used in the selection of the participants. The following data were transferred from the individual case record sheets at each study site to an Excel spreadsheet (see Supplementary Material 1): Age (years), sex (female/male), POC-ECO results (G1, negative; G2-G3, traces; G4-G10, positive), number of S. mansoni eggs per KK slide, number of S. mansoni eggs and the amount of stool (grams) tested with HTX. Only data from fully compliant participants were analyzed. Test results (positive/negative) per site are given as absolute numbers and percentages with 95% confidence intervals (CIs). Non-overlapping CIs indicate a significant difference between the percentages compared. For overlapping CIs, the CI was calculated for the difference between the percentages (https://www.statology. org/confidence-interval-difference-in-proportions-calculator/, accessed March 24, 2022), and the difference was considered significant if the resulting CI did not contain zero 13 . POC-ECO positivity was calculated by classifying traces as positive (t+) or negative (t-). KK and HTX positivity was calculated separately and in combination (KK + HTX) ( Table 1). Performance parameters of POC-ECO (t+ and t-) and KK were estimated using the MedCalc Diagnostic Test Evaluation Calculator (https://www.medcalc.org/ calc/diagnostic_test.php, accessed March 24, 2022), with HTX used as a reference for infection status (see Supplementary Material 2). Figure 2 displays the accuracy of POC-ECO t+, POC-ECO t-, and KK, which was calculated as the overall probability that HTX-negative and positive cases were correctly classified and given by the formula: specificity × prevalence + specificity × (1 − prevalence).
Of the 1,895 fully compliant participants, 462 (24.4%) were egg-positive at KK + HTX. The rate of heavy-intensity infections was 0.6%, as there were 12 participants with ≥400 eggs per gram (epg) in feces at KK: Five in Sempre Viva and seven in Jaguaritira. In Bom Jesus, a non-endemic locality, the only egg-positive case was confirmed as allochthonous. POC-ECO t+ yielded significantly higher positivity rates than KK + HTX in all study sites except Palmital. POC-ECO t-yielded significantly higher positivity rates than KK + HTX in Gavião and Estreito de Miralta. POC-ECO t+ positivity rates were significantly higher than POC-ECO t-in all study sites except Palmital. Egg-positivity rates were significantly different between KK and HTX in Jaguaritira, Cajueirinho, and Itaquara ( Table 1). The accuracy of POC-ECO t+ was significantly 1: Results from POC-ECO, Helmintex (HTX), and Kato-Katz (KK) from eight sites ranked by KK (one sample, two slides) positivity for Schistosoma mansoni. Urine was tested using POC-ECO batch 180907091 (except Palmital, where batch 201806011 was used), with traces scored as positive (t+) or negative (t-). The combined result of HTX and KK (KK+HTX) was used as a reference for infection status. N, the total number of fully compliant subjects; n, number of subjects by outcome (positive/ negative); CI, confidence interval. Non-overlapping CIs indicate a significant difference between percentages at an alpha level of 0.05 (5%).  lower than that of KK, except in Palmital and Jaguaritira; that of POC-ECO t-was significantly lower than that of KK, except in Jaguaritira; and that of POC-ECO t+ was significantly lower than that of POC-ECO t-in Gavião, Itaquara, Cajueirinho, and Sempre Viva. At none of the study sites was the accuracy of POC-ECO (either t+ or t-) significantly higher than that of KK (Figure 2).

POC-ECO t+ did not reliably reflect individual infection status
(positive or negative) as determined by a robust egg detection reference, which compromised the correct identification of infected and uninfected individuals. Additionally, it failed to reliably estimate the prevalence of infection in low-to-moderate prevalence settings (<50% by KK, according to the WHO 7 ). Community-wide POC-ECO t+ positivity in Gavião (69.2%) far exceeded the threshold of 30% using the POC-CCA assay recommended by the WHO 7 for classifying low-prevalence settings. The POC-ECO t+ positivity rate among fully compliant children aged 6-15 years was 82.9% (29 of 35). This would lead Gavião to a biannual MDA regimen, as it would be classified as a high-prevalence area under the WHO 7 . In Bom Jesus, the only egg-positive case was detected by KK, HTX, and POC-ECO t+. However, the latter yielded 34 positives, resulting in a positivity rate of 13.8% in the community as a whole and 21.7% (13 out of 60) in the 6-15 age group; this POC-ECO t+ result would incorrectly classify Bom Jesus as eligible for MDA according to the WHO 7 . Also, in Estreito de Miralta, only two of the 98 participants were positive for S. mansoni eggs, and POC-ECO t+ yielded 42 positive results. In contrast, in Palmital, there were four egg positives among the 97 participants, and POC-ECO t+ yielded only two positives (Supplementary materials 1 and 2). Because similar epidemiological conditions prevailed at Estreito de Miralta and Palmital, the discrepancy in POC-ECO t+ performance between the two sites could be due to the different POC-ECO batches used, as reported by Viana et al. 14 . Graeff-Teixeira et al. 9 and Favre et al. 10 found high levels of trace results with batch 180907091, which may explain the significantly higher positivity of POC-ECO t+ versus POC-ECO t-at all sites except Palmital, where batch 201806011 was used. However, the manufacturer's instructions for use state that POC-ECO t-is not suitable for diagnosis because the presence of two lines in the result window, regardless of the intensity of the test line, should be considered a reactive result. Overall, the sensitivity (true-positive rate) of POC-ECO t+ was over 80% (80.2%; 95% CI:76.1% -83.9%), while the specificity (true-negative rate) was less than 60% (55.0%; 95% CI:52.4% -57.6%). The high false positives of POC-ECO t+ found at sites of varying prevalence confirm previous results from a non-endemic area 9 and reiterate concerns by the WHO 15 regarding the reliability of current formulations of the test.
The MoH is currently updating the Schistosomiasis Action Plan 9 in accordance with the WHO targets and sub-targets for the elimination of S. mansoni as a public health problem by 2030 2 and adapting the recommendations of the new WHO guideline 7 to a country-specific context. After decades of community-based interventions by PCE, endemic areas have achieved a very low overall rate of heavy-intensity infections, as noted in this study. Of the 9.15 million KK tests conducted by the PCE from 2008 to 2017, only 27,149 (0.3%) had more than 400 S. mansoni epg (http://tabnet.datasus.gov.br/cgi/deftohtm.exe?sinan/pce/cnv/ pcebr.def, accessed March 24, 2022). In this way, the country will most likely succeed in validating schistosomiasis elimination as a public health problem (currently defined as < 1% proportion of heavy-intensity infections 2 ) in due time. Since the target moves on to interrupting transmission, verification will depend on developing a highly accurate diagnostic test, which is not currently available 2 . The WHO 7 proposes a verification framework that involves the use of a high-sensitivity test followed by a high-specificity confirmatory test. If this two-step process was employed in this study, the use of POC-ECO t+ as the first step would reveal 1,002 positive cases, 182 of which would be confirmed by KK; this process would miss 280 (60.6%) of the 462 participants who tested positive at KK + HTX (see Supplementary Material 1). The unsatisfactory performance of POC-ECO in areas with low egg burden has been reported in the manufacturer's package insert 001/2018 edition 10 . In conclusion, POC-ECO is not currently recommended as a diagnostic tool for monitoring and evaluating control measures to eliminate S. mansoni infection in Brazil.